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Introduction 


We  report  on  an  analysis  of  the  costs  and  benefits  of  the  Advanced  Technology 
Ordnance  Surveillance  (ATOS)  program  undertaken  by  the  Department  of  Defense 
(DoD),  initially  through  its  office  of  Advanced  Concept  Technology  Demonstration 
(ACTD).  The  ATOS  program  involves  fielding  Radio  Frequency  Identification  / 
MicroElectroMechanical  System  (RFID  /  MEMS)  technology.  RFID  technology  consists 
of  small  radio  tags  that  transmit  a  radio  signal  identifying  the  tagged  item,  often  through 
an  electronic  product  code  (EPC,  analogous  to  an  Universal  Product  Code,  or  UPC). 
RFID  applications  in  inventory  management  provide  one  kind  of  Automated 
Identification  Technology  (AIT)  to  facilitate  Total  Asset  Visibility  (TAV).  Such 
visibility  has  many  benefits,  such  as  reducing  inventory  shrinkage  and  facilitating  better 
customer  service  by  tracking  customer  orders.  RFID/MEMS  extend  RFID  by  providing 
more  information  than  just  the  identity  of  the  tagged  item.  In  the  case  of  the  ATOS 
program,  the  MEMS  technology  provides  information  on  the  temperature,  gravity,  and 
humidity  experienced  by  the  tagged  items.  These  factors  are  important  because  they 
correlate  with  the  useful  life  and  reliability  of  ordnance. 

DoD’s  ACTD  office  expedites  promising  technological  solutions  through  the 

onerously  long  federal  acquisition  process,  to  field  especially  promising  technology  more 

quickly.  Although  ACTD  projects  that  ‘clear  the  hurdles’  can  field  technology  in 

substantially  less  time,  the  hurdles  remain  high  -  the  large  majority  of  programs  proposed 

to  this  office  are  rejected.  One  of  the  key  steps  in  getting  ACTD  approval  is  a  compelling 

business  case  for  the  technology  (actually  this  is  multiple  steps,  as  the  business  case  must 
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be  refined  and  reconfirmed  at  various  stages).  A  large  consulting  organization  that 
initially  worked  with  the  ATOS  program  office  estimated  a  return  on  investment  (ROI) 
over  1000%  from  implementing  RFID/MEMS  across  the  Navy. 

This  high  ROI  estimated  for  ATOS  seems  to  follow  a  common  pattern  that  should  be 
viewed  with  skepticism  given  the  current  enthusiasm  for  RFID.  This  technology  may  be 
viewed  as  an  operations  technology  like  robotics,  flexible  manufacturing,  materials 
requirements  planning,  and  enterprise  resource  planning.  Early  reports  of  the  value  of 
such  technologies  often  fail  to  be  sufficiently  critical.  In  examining  management 
fashions,  the  process  has  been  documented  by  Abrahamson  and  Fairchild  (1999).  The 
typical  pattern  in  the  literature  is  unrestrained  exuberance,  followed  by  hostile 
disillusionment,  tapering  into  a  balanced  analysis  just  as  the  fashion  dies. 

We  think  the  same  pattern  can  be  said  to  apply  to  the  implementation  of  operations 
technologies.  The  objection  may  be  raised  that  technologies  must  have  a  core  observable 
functionality,  while  management  fashion  is  more  ephemeral  because  it  lacks  tangible 
substance.  But  as  Abrahamson  (1996)  has  pointed  out,  management  fashion  itself 
persists  because  it  is  not  only  a  ‘sociopsychologicaF  phenomenon,  but  also  a 
‘technoeconomic’  one.  The  process  of  management  fashion  itself  helps  winnow  the 
wheat  from  the  chaff,  and  establish  the  boundary  conditions  where  ‘technoeconomic’ 
benefit  ends,  and  ‘sociopsychologicaF  enthusiasm  begins.  Each  of  the  technologies 
listed  in  the  previous  paragraph  has  proven  its  value  in  some  organizations,  at  some  time. 
Likewise,  RFID  is  a  technology  that  has  undoubtedly  yielded  some  firms  substantial 
benefits.  However,  we  doubt  that  it  can  work  for  all  organizations,  all  the  time.  And  the 
current  exuberance  for  RFID  technology  should,  we  think,  give  cause  for  special  concern. 
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Our  independent  analysis  was  undertaken  as  a  part  of  the  process  of  getting  the  ATOS 
program  approved.  The  ATOS  office  felt  the  1000%  ROI  estimate  was  optimistic  and 
wanted  to  conduct  another  independent  ROI  analysis.  In  addition,  they  were  interested  in 
developing  a  risk  analysis  around  the  ROI,  to  understand  not  just  the  expected  rate  of 
return  but  also  the  potential  risk  of  receiving  a  lower  return.  Finally,  they  were  interested 
in  a  sensitivity  analysis  that  would  show  which  factors  most  influenced  the  return  on 
investment,  to  better  understand  the  impact  of  mis-estimates  of  costs  and  benefits. 

RFID  Valuation 

The  costs  and  benefits  of  RFID  have  been  the  subject  of  a  great  deal  of  recent 
attention  in  the  popular  press  (e.g.,  Covert,  2004).  The  primary  consequence  of  RFID  is, 
of  course,  better  inventory  accuracy.  This  has  financial  implications  in  several  areas, 
including  reduced  shortage  costs,  holding  costs,  handling  costs  for  missing  items  and  the 
cost  for  not-detecting  missing  or  unsaleable  items  in  the  incoming  delivery  (Fleisch  and 
Tellkamp,  In  Press).  In  addition,  labor  cost  may  be  reduced  as  the  requirements  for 
physical  count  and  investigating  the  causes  of  inaccuracy  are  reduced.  For  the 
RFID/MEMS  application  we  investigate,  the  cost  of  demilitarizing  ordnance  that  has 
become  too  unreliable  or  volatile  may  be  avoided  by  better  tracking  environmental 
conditions,  which  correlate  with  reliability  and  volatility,  and  using  ordnance  first  (for 
e.g.,  target  practice)  that  has  the  greatest  chance  of  becoming  obsolete  or  unreliable. 

However,  not  all  of  the  benefits  of  RFID  are  easily  quantifiable,  as  is  true  of  any 
advanced  operational  technology.  The  application  of  technology  to  facilitate  logistics 
and  operations  is  as  old  as  the  industrial  revolution.  However,  the  recognition  that  such 
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technology  is  difficult  to  value  using  traditional  cost  accounting  methods  is  more  recent 
and  coincides  with  the  widespread  adoption  of  process  automation  and  computer 
integrated  operational  technology. 

In  an  early  work,  Kaplan  (1986)  tried  to  make  the  case  that  discounted  cash  flow 
analysis  might  still  be  usefully  applied  to  value  computer  integrated  technology,  but  that 
discount  rates  should  be  lower  than  those  applied  to  conventional  projects,  to  account  for 
strategic,  difficult  to  quantify  factors. 

The  valuation  of  advanced  technology  has  produced  an  enormous  literature.  A  recent 
comprehensive  bibliography  listed  over  200  papers  dealing  with  the  valuation  of 
advanced  manufacturing  technology  alone  (Raafat  2002).  Many  authors  since  Kaplan 
have  recommended  dealing  with  strategic,  qualitative  factors  more  explicitly.  Some  have 
recommended  estimating  the  value  of  those  factors  as  best  as  possible  and  explicitly 
including  them  in  the  financial  analysis  (Primrose  1991).  Others  have  recommended  a 
hybrid  approach,  in  which  qualitative  factors  are  considered  in  a  complementary  analysis 
(Kakati  and  Dhar  1991).  Empirical  work  has  found  that  many  firms  do  not  use 
sophisticated  valuation  techniques,  but  those  firms  using  a  hybrid  justification  approach 
tend  to  have  better  outcomes  with  technology  implementations  (Small  and  Chen  1997). 

The  qualitative  factors  most  often  cited  by  firms  evaluating  advanced  manufacturing 
technologies  include  flexibility,  learning,  quality,  reliability  and  safety  (Saleh,  Hacker  et 
al.  2001).  The  first  two  of  these  have  been  of  particular  importance  in  advanced 
manufacturing  technology.  Flexibility  is  a  key  benefit  behind  Flexible  Manufacturing 
technology  and  organizational  learning  is  often  cited  as  a  key  benefit  to  many  business 
software  technologies.  Because  of  this,  specialized  techniques  have  been  proposed  to 
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evaluate  those  factors:  researchers  have  attempted  to  capture  the  value  of  flexibility 
through  combined  simulation  and  linear  programming  approaches  (Ramesh  and 
Jayakumar  1997);  valuing  learning,  and  particularly  learning  and  knowledge  acquisition 
through  infonnation  technology,  has  been  the  focus  of  whole  new  methodologies,  such  as 
Fuzzy  Cognitive  Mapping  (Irani,  Sharif  et  al.  2002)  and  Knowledge  Value  Added 
(Housel  and  Bell  2001).  The  primary  qualitative  benefits  of  RFID,  however,  revolve 
around  quality  and  reliability;  the  unique  benefit  of  the  RFID/MEMS  application  we  are 
evaluating,  as  we  shall  show,  is  safety. 


Method 

The  cost-benefit  investigation  of  this  proposed  RFID/MEMS  implementation  used 
both  qualitative  and  quantitative  methods.  Qualitative  methods  included  a  factorial 
structure  for  the  non-cost  related  benefits  of  the  implementation;  quantitative  methods 
involved  a  traditional  ROI  analysis  to  assess  the  value  of  implementing  RFID, 
supplemented  with  a  sensitivity  and  risk  analysis  of  key  factors. 

Assessment  of  Qualitative  Benefits 

A  factorial  structure  for  the  qualitative  benefits  of  this  program  was  developed 
through  a  combination  of  case  based  methods  and  multi-criteria  decision  techniques. 
Extensive  unstructured  interviews  were  conducted  with  five  current  members  of  the 
ATOS  program  office  and  two  former  members.  Each  of  these  interviews  was  recorded 
and  then  transcribed  for  later  coding  and  analysis.  Follow-up  interviews  were  conducted 
with  two  of  the  participants  and  other  participants  responded  to  email  queries  about  their 
interview  responses.  Building  on  these  transcripts  and  existing  theory,  we  developed  a 
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set  of  qualitative  factors,  representing  benefits  obtainable  through  the  RFID/MEMS 
implementation. 

To  capture  the  utility  of  these  qualitative  factors,  we  conducted  a  pilot  survey  with 
five  potential  users  (managers)  of  this  technology  at  an  ordnance  depot  on  the  west  coast 
of  the  United  States.  These  initial  surveys  collected  open-ended,  unstructured  data  as 
well  as  utility  assessments  -  each  interview  was  time  consuming  for  the  participants  and 
the  researchers.  The  survey  instrument  contained  questions  to  assess  the  relative  utility 
of  the  factors.  In  addition,  we  asked  participants  to  assess  the  relative  benefits  of  (1) 
ATOS,  (2)  the  status  quo  (no  technology)  and  (3)  a  simple  bar  coding  scheme,  on  each  of 
the  factors.  A  swing  weighting  technique  was  used  to  assess  relative  utility  (von 
Winterfeldt  &  Edwards,  1986).  These  swing  weights  were  then  converted  to  simple 
multiple  attribute  ranks  and  input  into  a  multicriteria  decision  support  tool  to  obtain 
factor  and  alternative  rankings,  as  well  as  sensitivity  analyses  on  the  utility  assessments. 
While  swing  weights  are  not  intended  to  be  equivalent  to  simple  multi-attribute  ranks, 
recent  work  suggests  that  various  multiple  criteria  weighting  techniques  tend  to  produce 
convergent  results  (Poyhonen  &  Hamalainen,  2001). 

In  addition  to  the  utility  assessment,  the  survey  also  included  several  open  ended 
questions  to  understand  the  reasoning  behind  the  participants’  responses  and  to  insure 
that  we  had  not  left  out  important  qualitative  factors.  Completing  the  survey,  including 
training  in  the  relevant  multi-criteria  technique,  required  a  half-day  from  each  participant. 

What  we  report  here  was  originally  intended  as  a  pilot  experiment,  prior  to  a  more 
rigorous  examination  of  qualitative  factors  related  to  the  RFID/MEMS  implementation. 
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When  discussing  the  results,  we  will  examine  the  reasons  the  more  comprehensive 
analysis  was  not  undertaken  and  the  potential  consequences  of  the  omission. 

Assessment  of  ROI 

As  with  all  public  sector  investments,  there  is  no  revenue  stream  to  estimate  as  an 
outcome  of  the  investment.  Instead,  returns  for  public  sector  ROI  projects  are  often  taken 
from  cash  flow  changes  based  on  estimates  of  cost  reductions,  cost  avoidances  or  funding 
deficiency  reductions. 

Funding  deficiency  reductions  essentially  represent  an  opportunity  cost  to  current 
expenditures.  All  government  agencies,  and  many  government  programs,  have  expressed 
needs  that  remain  unfunded  by  congress.  Within  certain  (fairly  stringent)  limits,  a 
program  manager  may  be  able  to  apply  cost  avoidances  and  cost  reductions  to  meet 
unfunded  requirements.  To  the  extent  it  is  possible  to  value  those  unfunded 
requirements,  e.g.,  reductions  in  inventory,  The  ROI  might  be  estimated  by  the  value  of 
the  forgone  alternative  investment.  However,  this  research  takes  a  conservative  approach 
and  does  not  explicitly  examine  funding  deficiency  reductions  or  other  opportunity  costs. 

Cost  reduction  and  cost  avoidance  have  an  important  distinction:  cost  reduction 
represents  decreases  in  outlays  already  obligated;  cost  avoidance,  however,  eliminates 
anticipated  expenditures  that  have  not  been  obligated.  In  the  case  of  cost  avoidance,  the 
government  retains  the  option  to  avoid  the  cost  by  reducing  services,  which  confuses  ROI 
estimates:  can  cost  avoidance  be  fully  considered  a  benefit  of  current  investments  if  DoD 
could  choose  to  avoid  these  costs  in  other  ways?  The  distinction  is  not  clear  cut, 
however.  Funds  for  future  year  maintenance  of  existing  weapon  systems,  for  example, 
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are  not  obligated  in  many  instances;  reducing  anticipated  future  year  maintenance 
expense  might  be  treated  as  cost  avoidance.  However,  the  government  has  no  cost-free 
option  to  avoid  maintaining  these  weapon  systems,  or,  at  a  minimum,  de-militarizing  and 
disposing  of  these  systems  in  some  future  year.  Our  analysis  does  not  distinguish 
between  cost  reduction  and  cost  avoidance,  on  the  assumption  that  the  government  will 
not  otherwise  reduce  services  related  to  ordnance  storage  and  maintenance. 

The  ROI  calculation  we  report  is  based  on  the  standard  formula  for  the  Internal  Rate 
of  Return,  with  changes  in  expected  expenditures,  or  cash  flows,  taking  the  place  of 
revenue  -  cost.  That  is,  the  return  on  investment  will  be  calculated  as  the  discount  rate 
that  makes  the  net  present  value  (NPV)  of  cash  flow  changes  equal  to  zero: 

IRR  =  i  3  NPV  =  NE[cashflows]  /  i  =  0  (1) 

n 

In  addition  to  the  ROI,  we  will  also  report  the  NPV  using  the  recommended  discount  rate 
of  5%  for  Federal  Government  investments  in  technology  infrastructure. 

The  ROI  calculations  are  based  on  implementing  RFID  in  the  US  Navy  and  the  US 
Marine  Corps  only.  A  portion  of  US  Navy  and  US  Marine  Corps  ordnance  inventory  is 
stored,  at  least  temporarily,  by  other  agencies,  including  the  US  Army.  This  analysis 
ignores  the  impact  of  this  cross-organizational  handling  and  storage,  on  the  assumption 
that  tag  infonnation  will  either  be  tracked  by  the  other  agencies  (the  US  Army  had  one  of 
the  first  RFID  programs)  or  that  the  tag  information  can  be  obtained  in  a  timely  fashion 
when  custody  changes  hands. 
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Factors  significant  for  cost  avoidance  and  cost  reduction 

From  the  plethora  of  ordnance  related  costs,  the  costs  not  expected  to  be  impacted  by 
RFID/MEMS  insertion  are  excluded.  The  costs  that  are  expected  to  be  reduced  by 
RFID/MEMS  are: 

•  Inventory  Labor  -  Includes  all  costs  to  perform  execution,  reporting,  inspection, 
safety,  ammunition  management  and  accountability  review.  This  cost  is  included 
in  anticipation  that  applications  will  use  RFID/MEMS  data  to  automate  some 
portion  of  these  tasks. 

•  Causative  Research  -  Includes  all  costs  to  reconcile  inventory  reports  with 
physical  counts.  This  factor  is  included  in  anticipation  that  improvements  of 
asset  visibility  will  reduce  the  need  for  such  research. 

•  Transportation  -  Includes  all  costs  to  perform  second  destination  transportation, 
including  distribution,  redistribution,  surveillance,  maintenance,  and 
demilitarization.  This  is  included  in  the  expectation  that  some  portion  of  this  cost 
is  due  to  errors  in  ordering  or  handling,  and  that  this  portion  will  be  reduced  by 
the  introduction  of  RFID/MEMS  technology. 

•  Demilitarization,  Maintenance  and  Surveillance  (DMS)  -  Includes  the  cost  of 
personnel  to  perform  DMS,  as  well  as  variable  storage  expense  related  to  DMS. 
This  factor  is  included  in  anticipation  that  some  portion  of  this  cost  could  be 
avoided,  if  older  ordnance  could  be  used  first,  and  potentially  less  reliable 
ordnance  could  be  used  for  training. 
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A  set  of  cost  reduction  and  cost  avoidance  point  estimates  were  developed  by  a 
consulting  organization  for  the  ATOS  program  office,  and  these  estimates  formed  the 
basis  for  this  research.  Additional  interviews  with  subject  matter  experts  and  our  own 
expertise  in  Military  Budget  Accounting,  Operations  Management  and  Economics  were 
used  to  modify  the  point  estimates  (in  every  case  the  cost  savings  estimate  became  more 
conservative)  and  to  make  distributional  assumptions  around  the  point  estimates. 

Data  sources  and  subject  matter  experts  consulted  for  cost  avoidance  and  cost 
reduction  estimates  included:  a  Receipt,  Storage,  Stowage  &  Issue  (RSS&I)  program 
manager,  Weapons  Officers  at  Naval  Air  Stations;  Ship  Weapons  Officers;  Ammunition 
Management  and  Accountability  Review  (AMAR)  publications;  1996  Projected  RSS&I 
requirement  with  Fleet  Ownership;  1997  DoD  Joint  Operations  for  Explosives  Report; 
the  Defense  Transportation  Tracking  System  Program  Manager;  the  Navy  Supply  Corp 
Transportation  Director;  Weapons  Support  Facility  Public  Works  Coordinators  and 
Ordnance  Officers  at  Seal  Beach,  CA  and  Yorktown,  PA;  a  former  Navy  Surveillance 
Program  Manager;  personnel  at  the  Marine  Corps  Programs  Department  in  Fallbrook, 
CA;  and  Navy  Total  Asset  Visibility  Automated  Identification  Technology  Project  - 
Sidewinder  Touch  Memory  Button  Maintenance  Records. 

Cost  reduction  and  cost  avoidance  estimates  are  shown  in  Table  1.  No  benefits  are 
expected  in  the  three  years  following  the  study.  Benefits  begin  to  accrue  in  the  fourth 
year  as  the  RFID/MEMS  technology  is  deployed.  Full  deployment  is  expected  to  take 
four  years  and  benefits  are  expected  to  increase  as  deployment  goes  forward.  Costs  are 
expected  to  stabilize  after  the  technology  is  fully  deployed  to  those  shown  in  Table  1  for 
year  seven. 
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Table  1.  Cost  Reduction  and  Cost  Avoidance  Factors 
Constant  2003  dollars  (millions) 


Inventory 

Labor 

Causative 

Research 

Transportation 

DMS 

Total 

Status 

Quo 

54.4 

25.5 

6.8 

248.0 

334.7 

ATOS 

FY01 

54.4 

25.5 

6.8 

248.0 

334.7 

ATOS 

FY02 

54.4 

25.5 

6.8 

248.0 

334.7 

ATOS 

FY03 

54.4 

25.5 

6.8 

248.0 

334.7 

ATOS 

FY04 

49.4 

20.4 

5.4 

206.5 

281.7 

ATOS 

FY05 

44.4 

15.3 

4.1 

165.0 

228.8 

ATOS 

FY06 

39.3 

10.2 

2.7 

123.5 

175.7 

ATOS 

FY07 

34.3 

5.1 

1.4 

82.0 

122.8 

The  expected  investment  costs  required  to  deploy  the  RFID/MEMS  technology  are: 

•  Modification  of  Commercial-Off-The-Shelf  (COTS)  RFID/MEMS  -  The 
program  office  cost  to  design,  modify,  test,  and  demonstrate  an  inventory  / 
surveillance  system  based  on  COTS  technology. 

•  Procurement  -  The  cost  to  award  and  execute  a  contract  for  the  manufacture  of  all 
components  and  fielding  of  the  system,  Navy-wide,  including  preprocessors, 
portable  and  fixed  readers,  radio  frequency  extenders,  tags,  installation  and 
training  costs . 

The  consulting  organization  initially  assisting  the  ATOS  group  again  provided  point 
estimates  for  these  investment  costs,  and  we  conducted  additional  investigation.  Our 
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estimates  of  procurement  costs  were  somewhat  higher  than  the  initial  estimates. 
Procurement  and  modification  cost  estimates  are  shown  in  Table  2.  Procurement  and 


modification  costs  are  based  in  part  on  actual  costs  incurred  during  a  demonstration  at  the 
Indian  Head  Division,  Naval  Surface  Warfare  Center.  Other  data  sources  and  subject 
matter  experts  consulted  in  developing  investment  cost  estimates  include:  the 
Department  of  the  Navy  FY98/FY99  Biennial  Budget  Estimate  for  Operations  and 
Maintenance;  FY97  NAVSEA  Instruction  8023.7  -  Space  Utilization  and  Storage  of 
Explosives  and  Inert  Ammunition;  and  price  quotes  from  three  manufacturers. 


Table  2.  RFID/MEMS  Investment  Costs 
Constant  2003  Dollars  (millions) 


COTS  Procurement 
Modification 


FY01 

5.3 

0 

FY02 

4.2 

0 

FY03 

4.6 

0 

FY04 

0 

7.3 

FY05 

0 

7.3 

FY06 

0 

7.3 

FY07 

0 

7.3 

FY08 

0 

4.5 

Although  some  systems  development  is  also  required  to  exploit  the  RFID/MEMS 
data  that  the  ATOS  implementation  will  generate,  these  costs  were  not  estimated  directly, 
beyond  the  cost  of  designing  and  implementing  a  database  and  application  programming 
interface  (API)  for  the  RFID/MEMS  data.  This  primarily  reflects  a  large  number  of 
software  redesign  efforts  already  underway;  it  is  anticipated  that,  given  a  well  designed 
data  base  and  API,  already  planned  systems  could  incorporate  and  use  RFID/MEMS  data. 
As  mapping  these  system  development  efforts  and  any  incremental  cost  they  might  incur 
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to  acquire  RFID/MEMS  data  was  beyond  the  scope  of  this  project,  we  chose  to  implicitly 
represent  this  factor  by  modeling  variance  in  the  cost  reduction  associated  with 
RFID/MEMS.  That  is,  we  modeled  additional  software  development  efforts  that  might 
be  required  as  a  risk  of  less  cost  reduction  and  cost  avoidance. 

Sensitivity  and  Risk  Analysis  Elements  and  Scenarios 

The  sensitivity  analysis  was  conducted  by  varying  key  elements  of  four  factors:  1) 
procurement  and  COTS  modification,  2)  status  quo  cost  reduction  and  avoidance,  3) 
technology  obsolescence,  and  4)  implementation  schedule  and  training.  Each  of  these 
factors  was  changed  in  turn,  ceteris  paribus,  constituting  an  analysis  of  four  separate 
scenarios.  The  risk  analysis  was  conducted  by  first  determining  appropriate  distributions 
for  key  elements  of  the  four  factors  listed  above,  then  using  a  Monte-Carlo  simulation  of 
1,000  trials  to  detennine  the  5th  and  95th  percentiles  of  the  ROI  and  NPV  that  could  be 
expected.  A  risk  analysis  was  conducted  for  the  base  case  and  in  each  of  the  four 
scenarios  used  for  sensitivity  analysis. 

For  the  first  (procurement)  scenario,  the  following  elements  were  all  increased  by 
25%  from  their  values  in  the  base  case:  COTS  modification  cost;  usable  magazines  per 
major  (and  minor)  weapons  depot  site;  amount  of  ordnance  items  (and  cases)  to  be 
tagged;  unit  costs  of  fixed  readers,  portable  readers,  hand-held  readers  and  tags;  number 
of  frequency  extenders  required;  the  number  of  hand-held  and  portable  readers  required 
per  site;  the  per  site  hardware  and  software  installation  costs;  and  all  hardware 
replacement  rates. 

For  the  second  (cost  reduction  and  avoidance)  scenario,  the  following  elements  were 
all  decreased  by  25%  from  their  values  in  the  base  case:  reduction  in  cost  (below  status 
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quo)  of  labor  for  reporting,  causative  research,  and  DMS;  reduction  in  out  of  pocket  DMS 
holding  costs;  and  reduction  in  transportation  cost. 

The  third  scenario  addresses  the  issue  of  technology  obsolescence.  Based  on 
depreciation  rates,  and  rates  of  technological  change  in  related  industries,  we  felt  that  the 
20  year  life-cycle  that  the  consultants  initially  forecast  might  be  optimistic.  While  we  did 
not  make  a  systematic  effort  to  obtain  a  better  life-cycle  estimate,  we  did  want  to 
investigate  the  sensitivity  of  NPV  and  ROI  to  technology  obsolescence.  To  do  this,  we 
varied  two  parameters:  we  reduced  the  number  of  benefit  years  by  25%  from  20  to  15; 
and  we  increased  the  discount  rate  (to  account  for  added  risk)  by  25%  from  5%  to  6.25%. 

The  fourth  scenario  addresses  schedule  risk.  The  pilot  results  from  our  qualitative 
factor  utility  assessment  led  us  to  believe  that  the  implementation  time  line  and  cost 
estimates  for  this  technology  might  be  optimistic.  Again,  we  did  not  make  a  systematic 
effort  to  obtain  a  better  estimate,  but  instead  modeled  the  sensitivity  of  ROI  and  NPV  to 
changes  in  the  implementation  schedule.  To  do  this,  we  modified  the  planned  roll  out 
shown  in  Table  2  (25%  in  years  4-7)  to  18.75%  in  years  4-8  and  6%  in  year  9.  We  also 
increased  training  cost  estimates  by  inflating  both  the  estimated  training  hours  required 
and  the  number  of  personnel  who  require  training  by  25%. 

To  conduct  the  risk  analysis,  we  assigned  distributions  to  each  of  the  model’s  major 
elements.  For  most  cost  elements,  subject  matter  experts  estimated  minimum  and 
maximum  values  around  our  initial  cost  estimates  and  we  assumed  a  triangular 
distribution.  When  we  could  not  obtain  estimates  of  minimum  and  maximum  values,  we 
assumed  an  exponential  distribution  in  most  cases  (because  an  exponential  distribution 
requires  only  a  single  parameter).  A  few  variables  had  percentage  estimates  that  were  so 
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high  (e.g.,  80%  reduction  in  causative  research  labor  cost)  that  an  exponential  distribution 
yielded  unrealistically  large  potential  savings.  In  those  cases  we  assumed  a  triangular 
distribution,  with  a  narrow  range  of  variability  (10%)  around  the  estimated  mean. 
Distributional  assumptions  are  shown  in  Table  3,  except  for  a  few  variables  that  the 
ATOS  program  office  requested  we  keep  confidential  (e.g..  Tag  Cost). 


Table  3.  Distributional  Assumptions  for  Model  Elements  in  Risk  Analysis 


Variable 

Unit  Cost  of  Fixed  Readers 

Unit  Cost  of  Portable  Readers 

Unit  Cost  of  Hand-Held  Readers 

Unit  Cost  of  Frequency  Extenders 

Cost  of  H-ware  Installation  per  Site 

Avg.  Cost  for  Software  Installation  per  Site 

No.  of  Personnel  to  be  Trained 

No.  of  Hours  Required  per  Person 

Annual  Scheduled  Tag  Replacement 

Annual  Frequency  Extender  Failures 

Annual  Tag  Failures 

Annual  Fixed  Reader  Failures 

Annual  Portable  Reader  Failures 

Annual  Handheld  Reader  Failures 

Report  Labor  Savings  (%  of  SQ  Report  Costs) 

Causative  Research  Labor  Savings  (%  of  SQ) 
Transportation  Redistribution  Savings  (%SQ) 

DMS  Out  of  Pocket  Holding  Cost  Savings  (%  of  SQ) 
DMS  Surveillance  &  Maintenance  Labor  Savings  (%SQ) 


Distribution 

Mean 

Min 

Max 

Triangular 

$  1,000 

$  500 

$  1,500 

Triangular 

$  1,000 

$  800 

$  1,200 

Triangular 

$  2,500 

$  2,300 

$  2,700 

Triangular 

$  50 

$  48 

$  52 

Triangular 

$  7,400 

$  5,000 

$  10,000 

Triangular 

$  1,500 

$  1,350 

$  1,650 

Triangular 

1070 

930 

1130 

Triangular 

8 

12 

24 

Exponential 

20.0% 

Exponential 

3.0% 

Exponential 

0.5% 

Exponential 

1.0% 

Exponential 

5.0% 

Exponential 

5.0% 

Triangular 

75% 

65% 

85% 

Triangular 

80% 

70% 

90% 

Triangular 

80% 

70% 

90% 

Triangular 

50% 

40% 

60% 

Triangular 

60% 

50% 

70% 
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Results 


This  section  reviews  the  results  of  our  analyses,  beginning  with  the  multi-attribute 

factorial  structure  that  emerged  from  the  literature  and  unstructured  interviews,  followed 

by  the  ROI  and  NPV  risk  and  sensitivity  analysis. 

A  factorial  structure  of  the  benefits  of  RFID/MEMS 

Interviews  with  one  of  the  military  executives  who  started  the  program  that  evolved 

into  ATOS,  disclosed  that  the  program’s  original  intent  was  not  asset  visibility,  but  rather 

safety  improvements.  While  investigating  an  explosion  at  a  hunker  at  the  Indian  Head 

Naval  Base,  Maryland,  it  was  discovered  that  degradation  in  ordnance  containers  had 

allowed  propellant  to  escape  and  accumulate  in  the  hunker  until  it  spontaneously  ignited. 

While  there  were  no  fatalities  in  the  incident,  the  clean  up  cost  over  $2  million  dollars. 

An  article  on  RFID/MEMS  technology  came  to  the  attention  of  the  officer  in  charge  of 

the  investigation  (the  person  we  interviewed),  who  realized  that  automated  monitoring  of 

the  ordnance  would  have  prevented  the  accident.  Interview  subjects  from  the  ATOS 

program  office  uniformly  felt  that  safety  was  both  a  prime  motivator  and  a  significant 

benefit  from  implementing  this  RFID/MEMS  technology. 

Other  qualitative  factors  mentioned  in  our  literature  review  also  seem  relevant  here, 

in  particular  quality  and  reliability.  Subject  matter  experts  from  the  ATOS  program 

office  confirmed  the  importance  of  quality  and  reliability,  citing  the  impact  of 

maintaining  reliable  asset  visibility  and  improving  ordnance  quality  by  reducing 

obsolescence.  Making  ordnance  more  visible  on  a  global  basis  and  closely  tracking  the 

environmental  exposure  of  that  ordnance  would  improve  operational  performance.  This 

factor  was  labeled  Readiness  in  our  analysis.  Readiness  means  (among  other  things)  the 
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degree  to  which  assets  are  ready  to  be  used  for  military  contingencies:  the  percent  of 
assets  that  are  in  full  working  condition. 

Finally,  subject  matter  experts  felt  that  the  head  count  reduction  attainable  by 
implementing  RFID/MEMS  has  qualitative  benefits  beyond  those  measured  in  dollar 
tenns.  These  benefits  include  the  operational  value  obtained  by  re-assigning  personnel  to 
tasks  more  significant  than  tracing  misplaced  inventory  or  conducting  physical  inventory 
counts.  Not  only  are  these  routine  tasks  non-value-added,  they  are  disliked  by  those 
doomed  to  perfonn  them,  lowering  work  satisfaction  and  the  quality  of  work  life. 
Conversely,  the  learning  that  would  occur  for  those  that  deployed  the  RFID/MEMS 
technology  would  improve  their  craft  skills  and  the  job  enrichment  would  likely  increase 
satisfaction.  Thus,  a  head  count  reduction  associated  with  these  tasks  not  only  improves 
the  effectiveness  of  ordnance  management  by  freeing  resources  for  more  important  tasks, 
it  likely  improves  job  satisfaction  and  quality  of  work  life  (and  hence  retention;  Maertz  & 
Campion,  2004)  for  those  employees  engaged  in  ordnance  management. 

Thus,  a  complete  picture  of  RFID/MEMS  utility  needs  to  include  at  least  3  qualitative 
factors:  safety,  readiness,  and  manpower.  While  it  was  beyond  the  scope  of  this 
investigation  to  attempt  to  measure  these  qualitative  factors  across  the  DoD,  we  proposed 
to  develop  a  tool  that  the  ATOS  program  office,  or  individual  commands,  could  use  to 
assess  them,  especially  the  magnitude  of  the  perceived  benefits  relative  to  the  cost- 
reduction  benefit. 


17 


Figure  1.  Range  and  median  of  criteria  weights 
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Relative  importance  of  benefits  (criteria) 

The  criteria  weights  that  emerged  from  the  second  survey  of  potential  RFID/MEMS 
users  are  shown  in  Figure  1.  The  five  respondents  all  scored  either  safety  (3/5)  or 
readiness  (2/5)  as  the  most  important  criterion.  No  respondent  ranked  cost-reduction  as 
the  primary  criterion  and  only  one  subject  ranked  cost-reduction  as  the  second  most 
important  criterion.  These  results  are  discussed  below. 

Relative  utility  of  RFID/MEMS  versus  Bar-coding  or  the  Status  Quo 

Only  one  respondent  gave  RFID/MEMS  a  higher  utility  than  the  bar  code  system; 
two  respondents  ranked  it  below  the  status  quo.  Figure  2  shows  the  median  estimate  and 
associated  range  for  the  relative  utility  of  the  alternatives.  Based  on  responses  to  our 
open-ended  questions,  we  think  these  scores  reflect  factors  beyond  utility  assessment. 
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Figure  2.  Utility  of  Tracking  System  Alternatives 
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Several  respondents  brought  up  implementation  issues  during  open  ended 
discussions,  although  it  was  not  a  question  we  discussed  directly.  For  example,  one 
respondent  commented  that  RFID/MEMS  and  Bar-coding  were  “level”  because  of  the 
“human  factor”,  and  that  RFID/MEMS  “wouldn’t  work  everywhere  on  all  asset  types”. 

Respondents  also  volunteered  their  frustration  with  “similar”  technological 
implementations.  For  example,  one  respondent  suggested:  “Before  any  additional 
money  is  spent  on  new  technology  we  need  to  first  ensure  that  old  technology  is  being 
used.  Common  sense,  basic  warehousing,  and  management  oversight  are  the  keys 
regardless  of  what  type  of  whiz-bang  system  we  have”. 

Thus,  rather  than  merely  assessing  the  utility  of  successfully  implementing 
RFID/MEMS,  participants  seemed  to  confound  two  factors:  skepticism  regarding  the 
implementation  of  RFID/MEMS  and  a  sense  that  RFID/MEMS  doesn’t  address  the 
highest  priority  ordnance  management  issues. 
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ROI  estimates.  Risk  and  Sensitivity  Analyses 

Results  for  the  risk  and  sensitivity  analysis  are  summarized  in  Table  4.  Based  on  the 
costs  in  Tables  1  and  2,  and  using  a  5%discount  rate  and  a  20  year  expected  life,  we 
estimate  that  the  expected  ROI  for  this  RFID/MEMS  application  is  154.8%.  Using 
Monte  Carlo  simulation  and  the  distributional  assumptions  shown  in  Table  3,  Risk 
analysis  estimated  the  5th  and  95th  percentile  as  148%  and  165.6%,  respectively.  That  is, 
there  is  no  more  than  a  5%  probability  of  a  return  lower  than  148%  and  at  least  a  95% 
probability  that  the  return  will  be  lower  than  165.6%.  A  sample  graph  showing  the 
distribution  of  ROI  outcomes  for  the  base  case  is  shown  in  Figure  3.  The  expected  NPV 
was  $1,737  billion  dollars,  with  5th  and  95th  percentile  estimates  of  $1,556  billion  and 
$  1 .995  billion,  respectively. 

Figure  3.  Probability  Interval  for  Base  Case  NPV 
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The  first  sensitivity  analysis  dealt  with  procurement  costs.  As  detailed  in  our 
methods  section,  we  increased  key  procurement  elements  by  25%  to  investigate  the 
sensitivity  of  our  reported  returns  to  changes  in  procurement  costs.  The  expected  ROI  is 
137.8%,  with  5th  and  95th  percentile  estimates  of  130.2%  and  149.3%.  The  expected 
NPV  is  $1,721  billion,  with  5th  and  95th  percentile  estimates  of  $1,505  billion  and  $1,962 
billion.  The  insensitivity  of  NPV  to  changes  in  procurement  expenditures  is  discussed  in 
the  next  section. 

The  second  sensitivity  analysis  dealt  with  a  25%  reduction  in  key  elements  of  the  cost 
reduction  and  cost  avoidance  benefits.  The  expected  ROI  in  this  scenario  is  138.2%, 
while  the  5th  and  95th  percentile  estimates  are  128.4%  and  153.4%,  respectively.  The 
expected  NPV  is  $1,361  billion,  while  the  5th  and  95th  percentile  estimates  are  $1,157 
billion  and  $1,708  billion,  respectively. 

The  third  sensitivity  analysis  dealt  with  technology  obsolescence  and  a  shorter-than- 
expected  RFID/MEMS  technology  life  cycle.  The  expected  ROI  in  this  case  is  virtually 
unchanged  from  the  base  case  at  154.9%,  as  are  the  estimates  of  the  5th  and  95th 
percentiles.  The  estimated  NPV,  however,  was  reduced  to  $1,167  billion,  and  the  5th  and 
95th  percentile  estimates  were  $1,051  billion  and  $1,350  billion.  We  will  discuss  the 
reason  for  this  difference  in  outcome  measures  below. 

The  final  scenario  dealt  with  slippage  in  the  implementation  schedule  and  unexpected 
increases  in  training  cost  and  time.  In  this  case,  the  expected  ROI  was  reduced  to 
138.8%,  while  the  estimates  of  the  5th  and  95th  percentile  were  reduced  to  131.9%  and 
148.8%.  The  NPV  estimate  dropped  to  $1,657  billion,  with  estimates  of  the  5th  and  95th 
percentile  at  $1,467  and  $1,938  billion. 
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Table  4.  Risk  Analysis  of  ROI  and  NPV 
NPV  in  millions 


Scenario 

5%  Prob. 
NPV  <  X 

Mean 

NPV 

95%  Prob. 
NPV  <  X 

5%  Prob. 
ROI  <  X% 

Mean 

ROI% 

95%  Prob. 
ROI  <  X% 

Base  Case 

$1,556 

$1,737 

$1,995 

148.0% 

154.8% 

165.6% 

1 .  Procurement 

$1,505 

$1,721 

$1,962 

130.2% 

137.8% 

149.3% 

2.  Cost  Red.  & 
Avoidance 

$1,157 

$1,361 

$1,708 

128.4% 

138.2% 

153.4% 

3.  Tech.  Obs. 

$1,051 

$1,167 

$1,350 

148.2% 

154.9% 

165.2% 

4.  Schedule  Risk 

$1,467 

$1,657 

$1,938 

131.9% 

138.8% 

148.8% 

Discussion  and  Limitations 

Discussion  and  Synthesis  of  Results 

The  factorial  structure  of  benefits  that  we  derived  from  our  literature  review  and  the 
unstructured  interviews  seems  to  capture  the  important  benefits  to  be  obtained  from 
implementing  RFID/MEMS.  Our  subsequent  interviews  with  potential  technology  users 
did  not  uncover  any  additional  factors  and  showed  that  each  of  the  factors  we  had  derived 
was  valued  by  those  users. 

While  the  results  of  such  a  small  sample  have  quite  limited  generalizability,  for  these 
users  analyzing  only  the  potential  cost  reductions  would  not  be  sufficient.  The  difference 
in  criteria  weights  between  “soft”  and  “hard”  criteria  is  not  large  (e.g.,  28.2  for  safety 
compared  to  24.9  percent  for  cost).  However,  these  results  indicate  that  a  cost-benefit 
analysis  that  ignored  “soft”  criteria,  such  as  safety  and  readiness,  would  omit  both  the 
primary  benefit  and  the  majority  of  the  benefit  (i.e.,  over  50%  of  the  utility)  anticipated 
by  these  users. 
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If  one  assumes  that  ATOS  will  create  net  benefits  in  safety  and  readiness,  these  pilot 
results  also  imply  that  a  cost-benefit  analysis  which  justified  ATOS  on  “hard”  criteria 
alone  would  be  extremely  conservative,  significantly  understating  the  total  value  of  the 
program,  at  least  for  these  users. 

Nonetheless,  the  ATOS  program  office  chose  not  to  pursue  a  broader  utility  survey  at 
this  time,  and  chose  not  to  incorporate  soft-criteria  and  cost  data  together  in  a  systematic 
analysis  of  alternatives  with,  for  example,  a  Data  Envelopment  Analysis.  In  part,  this 
probably  reflects  that  the  output  from  the  cost  analysis  was  so  overwhelmingly  positive. 
Even  if  one  agrees  that  cost  captures  less  than  Vi  the  potential  utility,  when  the  estimated 
ROI  is  over  150%  and  the  estimated  NPV  is  over  one  billion  dollars,  it  is  difficult  to 
justify  the  efforts  to  analyze  additional  benefits. 

In  spite  of  the  large  returns  estimated  from  implementing  RFID/MEMS,  the  potential 
direct  beneficiaries  of  the  technology  indicated  significant  skepticism  and  a  sense  of 
working  from  the  wrong  priorities  in  our  interviews.  Their  comments  may  represent  a 
serious  barrier  to  implementing  RFID/MEMS.  The  extent  to  which  other  stakeholders 
have  similar  perceptions  is  impossible  to  judge  based  on  qualitative  responses  from  five 
participants  in  this  pilot  study.  But  if  there  is  widespread  skepticism  about  the 
implementation,  or  a  widespread  belief  that  resources  directed  to  RFID/MEMS  are  being 
poorly  used,  it  may  reduce  the  benefits  realized  from  the  system  or  even  prevent  its  full 
successful  implementation.  While  one  expects  that  employee  support  may  not  be  such  a 
critical  factor  in  a  hierarchical  military  environment,  work  in  socio-technical  systems 
suggests  that  employees  often  find  costly  covert  ways  to  express  themselves  when  they 
feel  they  cannot  voice  their  discontent  openly  (Graham,  1993). 
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We  proposed  that  the  ATOS  program  office  conduct  a  survey  of  implementation 
barriers  among  potential  RFID/MEMS  users,  to  fully  assess  the  resistance  to  the 
implementation  and  to  inform  educational  efforts  to  address  those  barriers.  However,  to 
our  knowledge  such  a  survey  is  not  planned  before  implementation.  This  may  be  because 
the  need  to  educate  lower-echelon  managers  about  the  priorities  and  values  behind 
decisions  is  not  recognized  in  the  military  organizations  targeted  for  ATOS 
implementation.  Indeed,  it  is  possible  that  such  an  educational  effort,  if  it  were  perceived 
as  a  ‘justification  of  orders’,  might  be  counterproductive  in  a  military  environment. 
There  was  some  evidence  that  ATOS  stakeholders  felt  an  analysis  of  barriers  might  do 
more  harm  than  good;  for  example,  on  seeing  the  results  of  our  survey  indicating 
resistance  at  specific  locations,  one  stakeholder  responded  by  saying  “we’ll  make  sure 
and  implement  it  there  first.” 

To  incorporate  the  effects  of  institutional  resistance  to  RFID  implementation,  our 
sensitivity  analysis  did  include  a  scenario  in  which  implementation  is  delayed  and 
training  costs  increase  (Scenario  4  in  Table  4).  We  do  not  claim  that  our  schedule  delays 
and  training  cost  increases  in  any  way  represent  a  ‘worst  case’  scenario,  but  the  results  do 
indicate  that  returns  are  fairly  insensitive  to  changes  in  implementation  schedule.  One 
could  argue  that  a  $100  million  reduction  in  NPV  with  a  25%  longer  implementation  and 
25%  increased  training  cost  is  quite  significant,  however,  this  is  only  (roughly)  a  5% 
reduction  in  NPV.  So,  while  the  dollar  impact  is  large  (and  potentially  worth  further 
investigation),  the  expected  return  seems  fairly  robust  against  moderate  difficulties  in 
implementation. 
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Another  interesting  result  of  our  sensitivity  analysis  was  the  impact  of  technological 
obsolescence.  While  a  20  year  life  span  seems  optimistic  for  this  kind  of  technology, 
reducing  that  life  span  to  15  years  produced  virtually  no  change  in  ROI.  However, 
reducing  the  life  span  to  15  years  and  increasing  the  discount  rate  to  6.25%  produced  a 
fairly  substantial  reduction  in  expected  NPV  estimates,  from  $1,737  billion  to  $1,167 
billion  (a  33%  reduction).  The  similarities  in  ROI  and  differences  in  NPV  can  be 
explained,  in  part,  by  noting  that  the  increase  in  the  discount  rate  does  not  affect  ROI, 
since  the  IRR  is  a  discount  rate,  but  significantly  reduces  the  NPV.  Similarly,  the 
reduction  in  lifecycle  does  not  affect  ROI  because  returns  in  the  out  years  are  virtually 
meaningless  with  IRRs  ranging  from  14.8%  to  16.6%;  lifecycle  reductions  have  a  bigger 
impact  on  NPV  with  a  6.75%  discount  rate.  (We  verified  that  both  the  lifecycle  change 
and  the  discount  rate  change  had  an  impact  on  NPV.)  The  IRR  is  high,  of  course, 
because  the  dollar  value  of  the  benefits  is  so  much  larger  than  the  dollar  value  of  the 
investments.  This  is  a  known  issue  with  IRR  as  an  investment  measure  and  illustrates  the 
importance  of  using  more  than  one  measure  of  return  on  investment. 

The  importance  of  having  both  measures  can  also  be  seen  in  the  analysis  of  our  first 
scenario,  in  which  procurement  costs  increased.  In  this  case,  the  impact  of  a  25% 
procurement  cost  increase  was  an  11%  reduction  in  the  ROI  estimate,  but  only  a  1% 
reduction  in  NPV.  The  difference  can  again  be  seen  in  the  relative  importance  of  early 
year  returns  in  the  ROI  calculation  with  a  15.5%  discount.  The  change  in  procurement 
cost  is  dwarfed  by  subsequent  returns  in  the  NPV  calculation  (with  a  5%  discount  rate). 

Finally,  given  the  sizeable  cost  reductions  expected  from  this  RFID/MEMS 
implementation,  the  most  important  sensitivity  analysis  may  be  scenario  two,  which 
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investigated  reductions  in  anticipated  benefits.  This  scenario  is  doubly  important  because 
it  is  the  only  sensitivity  assessing  simultaneous  cost  reduction  initiatives,  which  may 
reduce  the  target  of  opportunity  for  RFID/MEMS.  However,  even  25%  reductions  in  key 
benefit  elements  produced  only  an  1 1%  reduction  in  ROI,  and  a  21%  reduction  in  NPV. 
Hence,  estimates  of  return  are  sensitive  to  mis-estimates  of  cost  reduction,  but  even  a 
25%  mis-estimate  still  yields  an  NPV  over  one  billion  dollars,  and  a  ROI  over  100%. 
Limitations 

This  research  was  funded  by  in  part  by  stakeholders  with  a  vested  interest  in  the 
success  of  the  ATOS  program.  In  part,  that  compensation  paid  for  a  software  tool  we 
developed  to  allow  the  ATOS  program  office  to  independently  conduct  the  sort  of  risk 
and  sensitivity  analysis  reported  here.  The  funding  also  supported  some  of  the  analysis  to 
develop  the  results  we  report  here.  While  our  sponsor  sought  an  objective,  unbiased 
analysis,  the  potential  for  unintended  bias  in  sponsored  research  is  well  understood  and 
remains  a  limitation  of  this  work. 

We  did  not  attempt  to  directly  model  the  application  development  costs  required  to 
exploit  the  RFID/MEMS  data.  We  dealt  with  this  omission  in  part  by  modeling  variance 
in  expected  cost  reductions,  but  a  limitation  of  this  research  is  that  these  costs  may  be 
greater  than  anticipated,  and  hence  reduce  the  reported  returns. 

Status  quo  inventory  costs  are  held  constant  in  the  risk  and  sensitivity  analysis.  As 
there  are  other  initiatives  being  implemented  to  reduce  these  costs,  or  to  avoid  future  cost 
growth,  this  is  not  a  conservative  approach.  Lacking  details  about  other  efforts 
underway,  excluding  status  quo  inventory  costs  from  the  risk  analysis  seems  reasonable, 
given  the  scope  of  our  analysis.  However,  it  remains  a  limitation  of  this  analysis  that  we 
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do  not  address  the  risk  of  continued  investment  in  other  inventory  projects  that  might 
reduce  the  cost  reduction  and  cost  avoidance  opportunities  in  the  status  quo. 

Conclusions  and  Summary 

We  have  developed  a  multiple  attribute  utility  structure  for  the  implementation  of  a 
major  RFID/MEMS  application,  in  which  safety,  manpower  issues,  and  readiness  (related 
to  what  is  called  safety  capacity  in  the  commercial  sector)  are  all  seen  as  important  non¬ 
cost  benefits.  Our  limited  analysis  of  this  utility  structure  shows  that  cost  reduction  may 
not  capture  all,  or  even  most  of  the  utility  to  be  derived  from  such  technology.  We  also 
saw  resistance  to  change  in  our  limited  sample,  potentially  representing  an 
implementation  barrier.  However,  our  cost  analysis  showed  that  this  RFID/MEMS 
application  should  produce  substantial  cost  savings,  and  our  sensitivity  analysis 
suggested  that  these  savings  were  robust  against  moderate  mis-estimates  from  our  subject 
matter  experts.  These  savings  may  justify  the  investment  without  weighing  qualitative 
factors.  When  the  cost  savings  are  not  as  clear  and  robust  as  they  are  in  this  case,  it 
remains  important  with  RFID,  as  with  other  operations  technologies,  to  be  able  to 
systematically  weigh  such  non-cost  benefits,  and  implementation  obstacles. 
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